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ABSOIEACT ^ 

Two pilot studies investigated the effects of using 
Instructional Math ,Play (IMP) Kits, pamphlets with which individuals 
m^y play the mathematical game EQUATIONS against a computer program. 
Twejity-nine junior high students 'in a high-ability mathematics cla$s 
completed varying numbers of the kits .in five 48-minute sessions 
during a two-week period; .^ten selected\ junior high school mathematics 
teachers worked through the entire series during a two-and-one-^half 
month period. Pre- and post-^tests^ere Idesigned to evaluate subjects' 
ability (1) to detect the ^relevance Of k particular id^a for solving 
a problem, and (2) to evaluate a mathematical expression involving 
that idea. Both groups made Significant! increases. A significant 
amount of "unencountered lealrning" was noted. It was suggested tR^ 
significant effects day be anticipated with less competent subjects ^ 
than tl^ose involved in the pi]^ot study. (Author/JBH) 
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INSTRUCTIONAL GA14ING AS A MEANS TO ACHIEVE SKILL 
IN SELECTING IDEAS RELEVANT FOR SOLVING A PROBLEM 

Layman E. Allen and J^an Ross 
University . of Mi\:higan 



Two pilot studies ini^estipated the effects o.f usinp I?^ (Instructional 
Math Play) Kits, pamphlets with which individuals mav plav the watheinatical 
game EQUATIONS against a computer proprap. The hiphly-braAched 105--kit 
sierle^ presents a total of ?1 mathematical ideas — five versions of each idea. 
tVenty-nine junior hieh school students in a hi^h-ability mathematics class 
completed varying numbers of the kits in -five A8^minute""sessions during a 
two-^week -period^ ten junior high school m*athematics teachers in a ^leadership 
training program worked throuj^h the entire series at theit individual paces 
duting a Xwo and one-half month period. Pre- and post-tests focusing on/ 
the IMP Kit ideas were ^designed bv the investlgs^tors to evaluate the 
subjects' abilities' to detect the relevance of a particular idea for purposes 
of solving a probleiS an^ to evaluate a mathematical T^^ression involving that 
idea. • 

On the relevance/s'electinp pretest the students achieved solutions tjo 
3.86 (mean) of the 21"problems: teachers achieved solutions to 8.50. On ^r^e 
posttest students increased their performance by 4.03 solutions to 7.^0, and 
teachers increased theirs " by 80 to 16.30. Both incTseases were significant 
at the* .0005 level. Students were also administere^straipht computation 
postt'ests involving the same 21 ideas: thev solved 6^7^ o^ the problems on 
the computation test, as compared with /their solution of AO?' t5f the problems 
on the relevance/selecting postpest. The magnitude of the chanj?es in per- 
formtoce after exposure to tfie I^^ Kits, suggests that significant effects 
may be anticipated x^ith somewhat'iess competent subiects than those invoiyed 
in the pilot studies. A surprising result was the significant amount of 
"uncncountered'^ learning" by students — that is, the number of ideas that a^ 
student missed on the pretest, did not encounter in poin^ throupb the I^fP 
Kits, but d.i^ cbrr^c.tly on the posttest. The distribution of the unencountered 
leamitjg rate suggests the usefulness of research directed to assessing 
^individual learning styles -(e.g. , specif ics--]eamers as contrasted with 
Renetalizers) in order to find more effective means of individualizing 
learning. The pilot studies pdint^to controlled experiments for further 
investipating whethej effective wavs can be devised to improve skill in 
Selecting relevant ideas for solving problems, ^ 
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INSTRUCTIONAL GAMING AS A MEANS TO ACHIEVE SKILL 
IN SELECTING IDEAS RELEVANT FOR SOLVING A PROBLEM* 

' Layman-E. Allen and Joan Ross 

* University of Michigan 

,/ INTRODUCTION ^ 



Any reasonable appraisal of the history of*" the Twentieth Century ■ 
would conclude that mankfnd desperately needg more rational J)roblem- 
solving. Our very existence is thre^t^ed to a degte|j that> it has 
never been before. A heavy burden must fee borne by tHfe term 'rational' 
if it is to have refere^ice td effee^tiye methods of coping with pro- 
blems of the magnit'ude /and complexity that^re faced on earth in the ; 
year 1974. Investigations aimed at improving the rationality of pro- 
blem-solving are of fundamenlbal importance. Studies in this category 
include, ^but are not limited to, (1) those that seek to clarify what 
rational problem-solving mu^t consist o'f in order to successfully deal 
with the important' cant enipor^ry problems of mankind, (2) those that 
encourage sigiiificant decision-makers to employ such rational problem- 
solving, and those that cast some light on how to educate problem- 
^firtsJLvers to fociis attention upon/those aspects of . a problem that ate 
' relevant for itfe foliation. As a tentative and preliminary character- 
ization of, what we mean by rational problem-solving in the context of 
the present study,, we stipulate the following: 

A rational/ approach to solving a problem is one that 
emphasizes t^e most i)owerfuI ideas 'and skills — that 
is, the indispensably relevant ideas and skills — that 
will enable a problem-solver to deal with a problem "±n 
a way that achieves his goals .to a maximum extent. 

^ The present study deals with ^six-^ ideas of elementary arithmetic 
and some problems for which these ideas and combinations of these ideas 
are not only relevant in the sense of being useful, but are also rele- 
vant in the sense of being indispensable for purposes of achieving 
solutions to the .problems. Compared- to the big problem^ of today, 
these problems are precise and simple. But even With relatively simple 
and precise prohiems involving only ideas of elementary arithmetic, the 
skill of a probletn-solver in selecting the indispensably relevant idea 
or ideas from*among the repertoire of ideas that he in some" sense 
"understands" is not a trivial skill. Thi's study represents a beginning 



* We gratefully acknowledge the indispensable relevance of the suppbrt 
and cooperation of the many dedicated educators who made these pilot 
stu-dies possi^ble. In particular we wish to thank Dr. FredericTc 
Schippert, Supervisor of Junior High Sdiool Mathematics, Detroit, MI. 
and Mrs. lone Gcfodman, Mathematics Teacher, Tappan Junior High 
School, Ann Arbor, MI.^ also Neil Muelleiq, Principal, Tappan Junior 
High School, and the ten Detroit Junior High School^mathematics 
teachers who participated in these studies. 
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effoirt in seeking to learn how such skill ifan be achieved by re.latively 
knowledgeable and competent persons with respect to relatively well- 
defined problem-solving. The hope, of cWirse, is that what is learned 
at this simple level can to some' significant extent be extended to 
more complex problems. ( 

. - ■ - ' ^ . .•;/ > N 

Instructional Math Play. (IMP) Kits y v . * 

This .preliminary report presents ^ome of the findings of ^ pair 
of pilot studies 'in the use of Innovative materials being developed 
that use instructional gaining and computer-assisted instruction to 
teach mathematics. These materials ate called IMP (Instructional Math 

;Play) Kits. The IMP Kits are 16-page pamphlets that, in effect, are 
simulations of a computer with which a player can play EQUATIONS: The 
Game of Creative Mathematics (Allen, 1964). In -a highly-branched 
learning program, the computer is programmed" to play, not as a good 
player., but as a good teacher. Each pamplilet allows the player as many 
as 2,000 to 3,000 different pathways, depending upon the sequence of 
moves that he makes, and each pamphlet is designed to present and 
emphasize one lesson in mathematics. A total of 21 different ideas arq 
presented an^^emphaslzed in the series of 105 IMP Kits developed thus 
far ~ five different versions of each idea. As a player goes through' 
an IMP Kit, each move he makes contributes to the generation of a unique 
code name, which the player looks up in a set of tables .in the IMP Kit 

.to discover what play the computer makes in response. Although there 
are thousands of different pathways that a player may take through a 
kit, there are only three types of pathways: 

1. ones in which the lesson of that kit' is not presented (in- which 
case the player is instructed to play the kit again and try^ 
some different moves) , 

2. ones in which the le^^ son is presented and the play indicates 
.that the player did ^ot understand the les^n (in whi^h case 

the player will movei^on to the next kit and encounter later 
kits that contain tfe same lesson) , and 

3. ones in, which the Ifsson is presented and the play indicates 
that ,the player did understand the lesson (in which case the 

^ player will be instructed to move on to the next ^it, but to 
^ skip the later kits that contain the same lesson). . 

METHODS 

The use of a series of 105 IMP Kits^was pre-piloted with five 
experienced players of EQUATIONS to determine the appropriate time to 
al.low for admirlistratipn of the learning program and of the pretests 
and posttests. , 

Subjects , 

Two different groups of subjects participated, in these pilot 
studies: 29' members of an eighth-grade class from Tappan Junior ^High 



School, Ann Arbor, MI.,. and 10 junior high school mathematics teachers 
ftom Detroit, MI. Each group \^as sele'cted as tKe mpst knowledgeable 
and highest-achieving group of its kind that was conveniently available 
to teat' these materials. 

The students were in one of the three top- track eighth-grade 
mathematics f^lasses at Tappan Junior High School where the total 
eighth-grade enrollment was 254, so all members of the experimental 
student group were in the top one-third in mathematics achievement in 
this school. Nearly 90 percent (26 of the 29 students) had parents 
engaged in professional or academic occupations. Ten of th^ students 
had at leas^t one year of experience in playing the EQUATIONS game in 
elementai:y school, and the other 19 had at least one month's experience 
as part of their studies in this eighth-grade class. There were 19 
girls and 10 boys in the group. ^ • 

The 10 'teachers were also a specially-selected group, having been 
chosen by the Detroit Dii^ector of Junior High School Mathematics In- 
struction for a year of academic leave during 1973-74 to enroll in a 
special leadership training program. Part of their training consisted 
of learning to teach *dther teachers how. to . introduce instructional 
gaming into the classroom. Two of the 10 h^ previously compJ.eted a' 
two-credit course ajt the Universitfy of Michigan in instructional 
gaming. All of them had learned to play. EQUATIONS to some extent be- 
fore this study began. There were six women and four men in the group. 

Experimental Treatment < ' 

The learning experience^^for both the student and teacher groups 
consisted of working through the set of 105 IMP Ki^ts. However, the 
method for doing so in the two groups was significantly different.; The 
student group spent six 48-minute classroom sessions spread over Bj two- 
week period playing EQUATIONS individually in the IMP Kits. The first 
session was devoted to instruction on how to use the IMP Kits^ and ito 
keep the necessary records and 'the remaining five sessions were de- 
voted toiiwork in the IMP Kits. Since the students worked ^t different 
rates, there were individual differences in the number of kits conlipl^- 
ted during the tw9^week period by each student. Only, two student^ were 
able to finish the set during the ali^tted time. Since the procedure 
allowed for; branching and skipping a niunber of kits, even these two did 
not work 'through all 105 kits; rather, they did thpse kits that their 
performance indicated were appropriate for them to do. Hence, only two 
of the students encountered in some sense all 21 of the mathematical 
ideas being presented in the lessons of this series of IMP Kits; the 
other 27 students encountered only some of the 21 ideas. At the con- 
clusion of the two-week period, most of the students who had not fin*- 
ished asked if arrangements could be made so that they could ^complete 
the kits. To gain some indication of how interested they really were 
in completing the kits, it was made somewhat inconvenient for them by 
requiring that they come in after school one day to pick up the kits to 
do over the Thanksgiving holiday. With this procedure 17 of the group 
picked up the kits, and 12 completed them at home.^ 



The teacher group also had. a one-session introduction, hnt all of 
their playing through the kits was done on their own time and extended 
6ver a period of two and one-half months, including the Christmas vaca- 
tion. They were receiving university course credits for their partici- 
pation in- tW Lead^tship Program, and the director indicated to them 
that their performance on these materials would be takigtv^^into account 
in grading them for the course. All ten of the teachers worked^ through 
to the final kit in the series, branphing and skipping as their perf or-- 
mances dictated., 

■ * ^ . ' ■ 

Dependent Variables ^ 

The effect of learning experience with the IMP Kits was measured 
by a specially-constructed series of tests targeted at the 21 mathe- 
matical ideas being presented. Each of the tests contained 25 items, 
four of which did. not involve IMP Kit ideas.- -Answers tg these four 
items were not considered in the evaluation. Versions of these tests 
were administered as pretests and posttests to determine whether there 
^as any change in scores associated with working through the IMP Kits.. 

These tests were deliberately '^designed to be. gfomewhat more dif- 
ficult than the customary achievement tests which d^al with the same * 
Ideas. They included items that required subjects to detect the in- 
dispensable relevance of an idea for- purposes of solving a problem", in 
Addition to knowing how to evaluate a mathematical expression iiivolving 
that idea. Understanding an idea in the latter sense is all that is 
Required in most of the items on customary achievement tests, but un- 
derstanding an i\iea in^ the former sense is equally' important for pur- 
poses of practical application of the ideas in real-world problem- 
solving. To indicate by way of exatnple the kinds of i^roblems included 
in these spedially-deslgiied tests, the instructions to the tests and 
several of the |f ems lllustratliig IMP Kit ideas from one of the tests 
are included h4j:6j : ; * ^ 

\ INSTRUCTIONS . 

By writing an X in the Yes or Norcolaimn, indicate whether or 
not all of- the numbers and operations in Column A can be appro- 
priately ordered and grouped to construct an expression equal 
to the number listed in Column'^B. 

If ydur answer is- Yes, write that expression in Column C. 
Column A Column B . Column C ' Yes Na 

EXAMPLE A , ^ . 

+ X 2 3 4 14 = 2 X (4+3 ) JL _ 

Operations and numbers are grouped by inserting parentheses as 
\ln EXAMPLE A. ' * ^ 

EiCAMPlE B ^ - ^ 

* 2 3 • 9 ' = 3*2 X __ 

^^ff^he * in Column A means "to the power of". Thus 3*2 indicates 
"ifVhat i9 usually written as 3^ 
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EXAMP1;.E C 



/ 2 9 '3 = 2/9 

The /.In Column A means "root of," 



X 



For purposes SIP answers here 
indicate what root you mean by putting a number to the ''lef t of _ 
the /. Put a 2 if you mean square ropt^ a 3 if yt)u mean cube 
root, etc. , ' ' . 



EXAMPLE D 
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^ + k 2 4 6 


11 


There should be no 


entry in 


, Column A 


Column B 


4. * / 6 6 7 


i 


5^. T * 1 2 9 


3 


6. * * 1 % 6 


6 


7. - - 1 3 5 


7 


8. V V 1 2 4 


■ ,.V8 . 


17. -7 V 1 3 5 


19 



Coliimn C 



Yes No 



Reader^ can gain some impression of the level of difficwJLty of these , 
items by working the problems themse|Lves. , The apj^rc^priale answers are 
include^ in the Appendix} In scoring th§se jt^stSi ^ Counted 
Correct for YES answers bply if an apjpl^o'pria|te' entry was made in 
Cpltmin d. I I ■ ' 

For^'the student group one Veifslo^L [of , these tests (Test A) was 
administered as a pretest to 15 ifiembers[ of the group on the Friday pre- 
ceding the two»-week jperiod, and another version (Test B) was adminis- 
tered to the other 14 members. On the Monday following the two-week 
period, the 15-memb^r grotip received Test B as a posttest, and the 14- 
member group recei%0d Tesd A. • Oiie week later ., after the 17 students had 
worked on: the remaining IMP ;kits at home, a post-posttest (Test C) was 
created and administered tp students whio had, not finished the 105-kit 
series during the five classroom periods. Students were allowed 20 
minutes; to complete these tests, and it was evident that this was ample 
time because when given a two-minute warning, ^practically all of the' 
stui^eiits immediately handed in their completed tests. In addition, 
because of some of the experience in this pilot run in the classroom, 
two special questionnaires were prepared and, administered during tl^e 
week following the two-week period^one by the class-room teacher direc- 
ted to how the students felt about this kind , of learning experience 
and the other by ^ the experimenters direcl^ed to what the students be- ' 
lieved accounted for their doing correctly on the posttest some pro- 
blems which they did incorrectly on the^prQtest and jwhich involved MP- 
Kit ideas that they did not encountet i^ their experience with the IMP 
Kits. Also a regular computation test involving the IMP ideas but not 
the relevance'^f eature was prepared and administered to the g^^oup 22 



days after the -two-week experimental period. The classroom teacher 
plans to administer additional post-posttests during the course of the 
remainder of the academic year. 

" Por^the teacher group, Test A« anH Test B were each admitrlstered 
as pretests to;half of them before- they began working 'through the IMP 
Kits, and after they finished the IMP Kits two and one--half. months ' 
later each five-person group received the other test as a posttest. 
The teacher's were allowed all the time they wished to complete the 
tests, and toofe from 35 to 55 minutes (an averagre of 46 minutes) to do 
them. 

k * '• .■ • ■ ■ 

|;/ RESULTS 

The data fromi .this piJLot- study suggest several, experiments to 
design and execute for whi'ch the results are likely to show highly 
significant clif f erences. Among these are: 

1, that students do better on the relevance/selecting tests 

after exposure to the IMP Kits than before, 
'2. that teachers do better after i^^^posure to the IMP Kits than 

before, : 

3. \^ that selected junior high school teachers do better both 

*before and after than selected junior high school students, 
arikK improve more in their performance than the* students by 
virtue of exposure to the IMP Kits, 

4. that the tests involving the skill of selecting the indis- 
^ pensably relevant ideas for solving the problem are harder 

than tests involving only computation with the same ideas, 

5. thati' $t:udents do better after exposure to some of the IMP . 
Kits Ijihan before even on IMP Kit ideas that they have not 
been 4^osed to, and 

^l^.^th^t'boy students do better than girl students in tlie ex- 

tent tio which they do better after exposure to the IMP Kits 
than before on ideas that neither have been exposed to in 
the IMP Kits, while the girls do better than the boys on 
* ideas (which both have been exposed to in the IMP Kits, 

The perfojrm^nces on the relevar^/selecting test of the students 
and the teachets both before and after exposure to the IMP Kits are 
summarized in table 1. On the pretest the students achieved solutions 
to just 3.86 (mean) of the 21 problems, "while the teachers- achieved 
solutions to 8.50. On the posttest the students increased their per- 
formance by 4.03 solutions to 7.90, and the teachers increased theire 

by 7.80 to' 16.^0.* - ' ^ . 

The statistical significance of the increase in performance by 
both the studerit group and the teacher grojup is summarized in TaMe 2. 
The mean increase of the students of 4.03 solutions is significant at 
the .0005 leveli (t = 9.13,)- The mean increase of the teachers of 7.80 
solutions is also significant at the .0005 level .(t « 8.19).. As judged 
by their performance, after working through the IMP Kits both groups 
understood about double what they had understood of the IMP Kit ideas 
beforehand. 



f . ' 

Table 1 ' 

Pretest, Posttest^ and Change Scores * 
on the Relevance/Selecting Test ' 

(21 Problems Involving the 21 IMP Kit Ideas) 





1 i } 

Pretest 


Posttest 


Change • 


- Students (A) 








^ n = 29 
a 


s . 






A 


J. DO 


T on 






1.71 , 


2.99 


2.38 


a 


2.91 


8.95 


6.04 


Teachers (B)' 
















" a 


8.50 


16,30 


7.8b 


\ ■ 


^.76 


3.23 


3.01 


'I 


7.61 


10.46 


9.07 , 



Table 2 



Change in Perf qrmance on Relevance /Selecting Test 
after Exposure to IMP Kit Ideas 

(Difference 7 Posttest Score - Pretest Score) 



Students 



Teachers 




n 
d 

^obs 

if icance 
Level 



29 
4.03 
9.11 

.0005 



^0005- ^28) = 3.67 

: V— 



-.0005 



10 ' 
7.80 
8.19 

.0005 
(9) = 4.78 
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Table 3 



Dlf feVen4e iVi PdrfoVmance Between Students and Teachers 
' • on Relevance /Selecting Test 



•4 





Pretest 


Posttest, 


Change 




2/62 


1.17 . 


1.50 


Significance 
Level 


.05 

F^g^(9,28) = 2.24 


not significant 


not significant 
^ ~ 


obs 


not appropriate 




""""^ 4.03' 


Significance 
Level 




^ .0005 

^ooo5^3^> = ^-58 


.0005 
^0005^37) = 3.58 ' 


obs 


5.00 






f 


11. A"? 






Significance 
Level 


.0005 


1 





The statistical significance' of the superior performance of the 
teachers compared to thd students both on the pretest and the posttest 
ahd ift the amount of Improvement is summarized in Table 3. That teach^ra 
do better than students is significant at the .0005 level in all three 
respects (pretest t* = 5.00, posttest t =7.51, and improvement t - 4.03). 
Since the assumption of equal variance on the student and teacher pretest 
scores is ,dis confirmed at the .05 level of significance (F, = 2.62), the 
t statistic is inappropriate; instead the Satterhwait^ app.roximatiftn of 
the Behrens-Fi'sher t* distribution is used to test the hypothesis of 
equal means for the teachers and students on pretest scores. (See 

Winer , pp . 41-44 . ) ' * , i / 

The comparative performances of the g,tudents on the relevance/ 
selecting test and the computation test along- with the statistical sig- 
nificance of the difference in scores ±s summarized in Table 4. One of 
the students had transferred fsom the class by the time that the compu- • 
tation test was administered, so a total of- 28 students was involved in 
the comparison on this pair of tests. -For the -computation test, 19.4 
(mean) of the students were correct over the 20 IMP Kit ideas compared. 
(One IMP Kit. idea that did not lend itself to presentation in the compu- 
tation test was not; used in--the comparison.) qn the relevance/selecting 
test, the number of students correct was only 11.3 (mean) per problem. 
Another way of stating it is that the students got 69% of the problems 
correct 'on the computation test, but only 40% on , the relevance/selecting 
test. The difference is significant at the .0005 level (t =5.17). 
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Table 4 



* Comparison of Nikber Students Who -Responded Correctly 
• to Problems Involving 20 IMP Kit Ideas Presented In 
(a) a Relevance/Selecting Test and (b) a Computation Test 



(28 Students) 



4: 





, Relevance/ 
Selecting ^ 
Test 


Cbmputatlon 
Test 


Difference 


• n »- zu 








• X. 


11.30 


\ 19.40 


8.10 


S 

X 


8.24 


•8.2^/ 


7.01 . 




- 69.61 


" ^8.02 


49'. 


t • 

Significance 
..-v.^ L(§vel 

5 — J 


4 


• 


' 5..I7' .. 

.0005 ^ 
t,9995(19)=3.8l 



iHie comparative performances of the boy students and the girl 
studeritg in learning ideas from the IMP Kits, both those ideas encoun- 
tered in their exposure to the kits an^ also those not ^n count e reef, 
,are summari^zed in ifable 5. The rate*' at which students learned ideas 
that they did not encounter in going through the IMP Kits Is here re- 
ferred to as their "uneijcountered learning rate." Onia. of' the most 
\ surprising results of this pilot study was* a significant amount (at 
\the ^.0005 levefl, t = 6.444) of such unencountered learning; the mean 
^Amencountered learning rate for the entire group of students was .132. 
The unencountered learning rate for 'each student is derived by dividing 
(a) the number of ideas such that the studfent both missed those ideas 
on the pretest and also failed to encounter them in going through the ^ 
IMP Kits^ into (b) the number oF those ideas that were correctly done 
on, the posttest. The encountered and pverall learning rates are de- 
rived similarly. These djefinition^ of the learning rates assure,^of 
course., that all effects will be in the positive direction. _ However , • 
by an even mofe stringent definition of unencountered learning that 
would allow for effects, in the negaj^ive direction, there" is still 4 
significant amount (at the .01 level, t = 2.841) of such unenppuijtered 
learning. See Table 6. .* 

'As would be expected the encountered .learning rate for the eiftire 
group was significantly highex (.0005 level, t = 6^685). than the unen- 
countered learning ^rate. The mean encountered learning , rate was . 404 
and the overall was .287. ^ . ' 
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Table 5 



Coinparls;;pns of Learning Rates (Encountered,* Unenc(;mntere^, ^ 
and Overall) of Boy* Students and Girl Students . 





Endounterdd 
Learning Rates 


Unencountered 
Learning Rates 


Overall 
'Learning Rates 


n 

A 

s 

X 

X 


Boys Girls Total 
. 9 20 29 
• O/l . 4ZU • *fU4 
.243 .211 .218 

.059 .044 .048 


Boys Girls Total 
9 20 29 

.108 .097 .107» 
.012 .009 .012 


Boys Girls Total 
9" ° 20 29 

V 

• Z7U •ZoD •Zo/ 

.128 .144 .137 
.016 .021^ .019 


t , ^ (Total 

ObS ; V 

p re-post) 

Significance 
Level 


' 9.990 

' * .0005 
^S«995^27)=3.690 


7 — - 

. 6.444 
.0005 

t 9995 (27)=3. 690 


11,2,64 
* * 

.0005' 

^9i95<27)=3.690 


P (Bdy- 

* Girl) 

Significance 
LeveJ. 


V 1.333 

A. 

NS . 

F .^g(8,19)=1.43 


1.219. 

■/ NS 
F yg(8,i9)'=1.43 - 


NS • ' 
F.^5,(8,19)'=1.43 


t , ^ (Boy- 

Girl) 

Significance 
. Level 


2.035 
.05 

t 35(2?) = 1.703 • 


2. 254 

^ .025 
t g^5(27)=2.052* 


^ .067 
NS" 

t^go(?7) = .256 



ts^^^Total Encountered- 
° . ' Unencountered) 



Significance 
Level 



6.685 
.0005 



t,„„(27) - 3.690 



There were also interesting differences in performance bett\reeh boys 
and girls. The girls bad a sigjiificantly C.05 levels borderline .025 
level, t = 2.035) higher ' encountered leamiilg rate (.420) than the boys 
(.371), while the boys had a slgnff leant ly. ( .025 ' level , t = 2.254) 
higher unencountered learning rate (.193) than the girls (.105). 

A definition of uneAcountered learning that would allow for an 
Indication of "unjLeaming" is reported in,. Table 6. The test results 
show that some students do proBlems involvirfg the same idea correctly bn 
the pretests ^nd incorrectly on. the posttests. Interpreted as '^hoise" 
and assuming that there is the same amount of noi^se in the^ data oft 
students that do problems incorrectly on the pretest 'and correctly on 
the posttest., there is a question as to what extent the indicated gai^s 
on the unencountered items can be attributed to ^oise, rather than to 
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"learning" in some sense. The more stringent definiticfn of unen- 
countered .leamirig (ULR ) discounts the first definition by the amount 
o^ noise. • ^ ' " 



ULR 



B^A 



s A+B+C 



, where A « number of items correct on pretest^ 
^: > ^and incorrect on posttest, 

, B viiumber of items incorrect on pretest 
and cprrect on posttest,^ 

C = number of items incorrect on both 
, . tests. * 



Table 6 / ' 

Significance of Stringent^ TJnencountered Lear^iing -Rate (ULR ) 
and Comparison of Perf ormanoes of Boy Students and Girl Students 







. ULR 

s 

, . ■ • N 


a 




Boys 


Girls 


4:otal 




9 


20 • 


29 . 


X : 
s 


.193 
-.168 


.061 
■ .10 8 'V 


.102 

« 

.123 


s2 : 


.0116 


.0116 


.0151 


- : . ^ 






^ . 



Significance 
Level 



•t.59(27) 



2.841 

.01 
2.473 



F (Boy-Girl) 



Significance 
Level 



1.004 
not significant 



F 75(8,19) = 1.42 



'^obs(^°y-Gi'^l) 



3.057 



Significance 
Level 



.005 



t 995(27) = 2.77.1 
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' * ■ V ' ■ * 

In additi^on to there still bein^ significant unencountered learning as 
measured by ULR , there was also significantly more of it by boys than 
girls.. Since none of the boy^' were incorrect on posttest* itCTis that 
they were correct on in ther pretest , while, some of the girls were, 
the differences in favor of the boys were eyen more significant as 
measured by ULR (.005 level, t*= 3.057). i 

To provide some further informa-tion about the surprising exis- 
tence of this' "unencountered learning/' the' 21 studetitB who exper- 
ienced it completed the following- op inionnaire : , * 

Name , • . ' ' * ; 



On the test you took before you played the- IMP Kits, you did not 
show that you could use this idea: 

(Example inserted here of the idea that this student. 

missed on pretest, did not encounter, but^ did correctly 

on posttesf) 

pn the test you took after you played *some of the IMP Kits, 
you showed that you cbuid use it. 

We are interested in the fact that you seem to have learned this 
idea during this time. However, the kits you played in class 
did riot present this idea. Please check the reason (or reasons) 
that best apply: ' T 

0' I learned it from another student. > . ^ 

1 I learned ±V in math class between the.txvro tests, but not 

' from the kits . ^ . ^ ' 
3 I guessed at it on the second ^st, and it tumejd out to 
be right . " 
11 I learned something from what I saw in the kits that helped 
me td do it on the second ^^test. 
8 Other . ■ : ' • . : 



The numbers in tjie blanks on the left side indicate the number of 
res^oijses to the right that x<rere given by students. 

f DISCUSSION 

-The^^^e^c^rimental treatment of wo rkingr through the* IMP Kits has a 
prpnouncedeJfBCtnapon^the-a^^^ problem-solving skills of highly 

competent junior high school students and teachers oil problems that 
require both<,knowledge of how to compute and also the skilL^.t)f selecting 
the relevant ideas for ^c^S^ng the problem. The magnitude of the 
effect suggests that significant effects may also be anticipated with 
somewhat less competent subjects t^an those in the pilot studies 
reported here. \ . 

The magnitude ot the change in performance after exposure to the 
IMP Kits, in combination with the large difference in performance *on 
the 'computation"' tes't as compared with performance on the relevance/ 
selecting test, Indic^es that the experimental, treatment ancf dependent 
variables used in these studies can provide an |jEfective test of . ' 
whether skill in selecting relevant ideas f&it solving a problem can be 
improved. The particular null hypothesis^lat can be tested is: 
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Working through IMP Kits is not an effective. 'V 
\ ' method for improving skills in selecting relevant 
ideas for solving arithmet±'Gal''^roMfems. 

A pair of interesting side issues aliso emerged: one /Sealing with 
generalization and the other with possible differences in learning 
styles of the two sexes. Through exposure to one set of ideas,* some 
students^ apparently learn something about a related set of ideas. For 
example, perusal of performance on various test items suggests that 
for some students learning something about exponentiation by working 
through the IMP Kits also , increases performance on problems involvitxg 
the root operation — even though tfne learner does not encounter the 
latter idea in the kits. If it turns out to be more generally the 
case that some groups of learners tend to generalize better in certain 
kinds of learning environments than other |^roups, it w£ll be of 
interest to study the attributes of those groups and those learning 
environn^ents that contribute to this ability. The association between 
sex and generalization dg^tected in this pilot study with the students 
suggests the usefulness of research devoted to determining attributes^ 
of learners* that will detect such strengths and differences in 
learning style more ef f ect|.vely Knowledge of such properties may 
well lead to more effective ways of individualizing learning. For / 
example, with respect to individualizing learning environments for 
gerieralizers an^ specifics-learners, perhaps some grouping according * 
to performance measured by the unencouhtered learning r^te — e.g.', - 
upper quartile and lower' quartile — would serve adequately. Such 
individualizing should certainly be acconjpanied by efforts to learn 
more about effective means of enabling all leaders to become strong 
generalizers. Hopefully, achieving greater skill in selecting 
relevant ideas for solving problems will help learners to generalize 
their experiences more effectively. 

The main results of these pilot studies point rather directly to 
experiments for investigating %he-«initial question raised in this 
report: Can effective ways for in^proving skill in selecting relevant 
ideas for solving problems be devised? Instructional gaming~or, to 
be more precise, simulation of ins tructipnal^ gaming' in the for^i of IMP 
Kits — may be an example of one ef f iactiv^* way .to improve such skill. 
In terms" of the* kind of data generated in these studies^ one measure 
of the lack of such skill would be a discrepancy between a subject's 
performance on the computatjpn test and his performance on the 
relevance /selecting test. The comparison of performances for each / 
subject on such tests would identify ideas that the subject does 
understand in the sense of b6itlg able to use those^ideas in making 
computations, but does i;iot understand in the sen^e of having sufficient 
skill' to detect the relevance of those ideas in solving, a somewhat , 
broader problem and selecting them to solve it. A pair of such teats 
administered before an experimental treatment would .provide a differ- 
ence score indicating the extent of the subjects* lack of skill in 
selecting relevant ideas; a pair of posttests, another difference 
score, indicating the extent of any change in such* skill associated 
with the experimental treatment. Such data were unfortunately not 
generated in these pilot studies; the agf»nda for future investigation 
will give high priority to remedying this deficiency. 
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' • Appendix , ' ■ 

A. Some Suggested Responds to 'iSampli Itetn^ f rom; Reievan(::iifi/Selectlng 
Test ' : . ■ [ ■ :•: ; ^ • ' ' 

\ Yes 1^ V 





Colvimn A 


Colxunn B 




Column C 


4. 


* / 6 6 7 


7 




6 ^7 * 6 


5. 


V * 1 2 9 


° 3 




9 * (lv2). 


6. 


* * 1 3 6 


6 




6 * (1*3) , 


7. 


- - 1.3 5 


' 7 




'3 - (1-5) . 


8. 


V V 1 2 4 


8 




2 V (1*4)^ 


17. 


- ^ 1 3 5 


19 






Ideas 


of the IMP 


Kits 







X 



1 



1. A + B - C =° A - (C - B) ; especially A + B =■ A - (0 - B) . 

2. A X B' V C' = A ^ (C V B'); espedially i X B' = A - (It B'). 

3. A = A * 1. ,{ 

4. 1 = 1 * A. . ■ 

5. 0 = 0 * A' . , ► 

6. A * (B + C) = (A * B) X (A * C) . 

7. A, * (B X C) = (A * B) '* C. 
1 T [A' * (B - C)] = A' * (C-B); especially 1 v A' = A' * (0-1) 
1 = A' * 0., ^ 

A X [B' * (C-D)] = A V [B' * .(D C) ] ; Jespecially 

A X B' = i V {B' * (0 - 1)]. ^ 

11. A / B = 'C if and only if C * A = B. - ^ 

12. A' / B = B * (1 V AM. ■ 

13. 0 = A' ^0. - ' 

14. (A T B') * C = (A * C) T (B' * C). 

15. A = 1 v/" A. 

16. . 1 = A' / 1. ^ . 

17. * A = B' / A * B' . 



8 
9 
10 



16 



'15 

; • ' --t • * ■ ' ; ■ ■ _ 

18-. 1 * .fU -\b) v/" C] - (B - A) v/" CTespeclally 

* • C - (0 - 1) v/" C. 

,19. (A' X B') v/" C - A' v/" (B*^/ C). V 
20. .A * (B' V C') = (C * B') v/"A; especially 

= A * B' - (r V B') / A. 

-21. [A' / (B* * C')] V D - [.(A' r C') vA'B'r v/" D. 

Notes: In standard notation, • 

a. A * B is A^, ' 

b. A / B is Vb, and 
■\ c. 2 aA B is vAB. ' ^ 

Variables with primes s>tand for non-zero n 



